Systems and methods for an automated cloud-based video surveillance system

ABSTRACT

Systems and methods for setting up a cloud-based video surveillance system with at least one computing device and at least one video camera in a local area network. The at least one computing device has an application program installed and is operable to find out the at least one video camera on the same local area network and connect the at least one video camera to a cloud platform. The video surveillance system is accessible via the at least one computing device locally or other computing device remotely. The video surveillance system is still at work when the at least one computing device is powered off.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application claims priority from a co-pending US application; it isa continuation-in-part of U.S. application Ser. No. 14/249,687 filedApr. 10, 2014, which is incorporated by reference herein in itsentirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to cloud-based systems and methods forconnecting a device with visual sensors to a cloud platform forsurveillance, storage, analytics and management.

2. Description of the Prior Art

It is known in the prior art that a video surveillance system can be setup at a location with a local recorder and server besides cameras. Thereis a big up-front investment for this type of video surveillance. Inrecent years, with anything as a service (XaaS) getting popular, videosurveillance systems are installed with minimum hardwarecomponents—cameras or other sensors. Everything else, such as streaming,storage, management, and analytics, is moved to the cloud. Theseservices are sold as a service by subscription with a monthly or yearlypayment. Still, it is a dedicated service and a big expense in a longterm.

Recent years, the vision of the Internet of Things (IoT) connects moreand more things to the internet and puts more intelligence and sensorsinto the connected systems, from industrial machines to home appliancesto automobiles. Apparently, there is a need for people to keep connectedwith everything they are concerned with no matter where they are andwhat they are doing. For example, an Xbox gamer may need to keep an eyeon a sleeping child while he is playing games. The present inventionprovides a technology to enable users to set up their own videosurveillance system on top of their existing smart devices. For example,gamers, TV viewers, drivers and others are able to set up a videosurveillance system within their homes or cars by themselves with theirgame consoles, smart TVs, smart cars, or other smart devices. Therefore,gamers, TV viewers, drivers can do what they want and still keepconnected with other things happening around them. There is prior artfor setting up video surveillance system, however, the videosurveillance system only works when the server is on. The presentinvention provides a technology to enable users to access the videosurveillance system remotely when the existing smart device is poweredoff.

As for future surveillance, the vision is that more and more devices areto be embedded with visual sensors and connected to the internet, whichcan be called the Internet of Things Videos (IoTV). Video and image dataas well as other sensory data are collected for monitoring and advancedanalytics and intelligent operation so that smart surveillance isprovided for various environments. For example, visual sensors can beembedded in thermostats, ovens, refrigerators, doorbells, toys, cars,and other smart devices.

Every smart device usually comes with an application program (“app”)developed by the device or appliance manufacturer. Currently, there isusually an app associated with a specific video device for remotemonitoring and communication. For example, Dropcam camera has its ownapp for cloud setup and remote view, and a Skybell video doorbell hasits own mobile app. There is a need for a universal application whichrecognizes all smart devices with in a local area network; thus, allsmart devices in the local area network can be connected to one cloudplatform and accessed via one app.

By way of example, prior art documents include:

U.S. Pat. No. 8,559,914 for “Interactive personal surveillance andsecurity (IPSS) system” by inventor Jones filed Jan. 16, 2009, describesan interactive personal surveillance and security (IPSS) system forusers carrying wireless communication devices. The system allows userscarrying these devices to automatically capture surveillanceinformation, have the information sent to one or more automated andremotely located surveillance (RLS) systems, and establish interactivityfor the verification of determining secure or dangerous environments,encounters, logging events, or other encounters or observations. ThisIPSS is describes to enhance security and surveillance by determining auser's activities, including (a.) the user travel method (car, bus,motorcycle, bike, snow skiing, skate boarding, etc.); (b.) the usermotion (walking, running, climbing, falling, standing, lying down,etc.); and (c.) the user location and the time of day or time allowanceof an activity. When user submits uploaded (or directly sent)surveillance information to the public server, the surveillance videos,images and/or audio includes at least one or more of these searchableareas, location, address, date and time, event name or category, and/orname describing video.

U.S. Pat. No. 8,311,983 for “Correlated media for distributed sources”by inventor Guzik filed Dec. 14, 2009 (related to U.S. Publications2010/0274816, 2011/0018998, 2013/0027552 and 2013/0039542) disclosesmethod embodiments associating an identifier along with correlatingmetadata such as date/timestamp and location. The identifier may then beused to associate data assets that are related to a particular incident.The identifier may be used as a group identifier on a web service orequivalent to promote sharing of related data assets. Additionalmetadata may be provided along with commentary and annotations. The dataassets may be further edited and post processed. Correlation can bebased on multiple metadata values. For example, multiple still photosmight be stored not only with date/time stamp metadata, but also withlocation metadata, possibly from a global positioning satellite (GPS)stamp. A software tool that collects all stored still photos takenwithin a window of time, for example during a security or policeresponse to a crime incident, and close to the scene of a crime, maycombine the photos of the incident into a sequence of pictures withwhich for investigation purposes. Here the correlation is both by timeand location, and the presentation is a non-composite simultaneousdisplay of different data assets. Correlating metadata can be based on aset of custom fields. For example, a set of video clips may be taggedwith an incident name. Consider three field police officers each in adifferent city and in a different time zone, recording videos and takingpictures at exactly at midnight on New Year's Day 2013. As a default, agroup may be identified to include all users with data files with thesame Event ID. A group may also be either a predefined or aself-selecting group, for example a set belonging to a security agency,or a set of all police officers belonging to the homicide division, oreven a set of officers seeking to share data regardless of if they arebellowing to an organized or unorganized group.

U.S. Pat. No. 7,379,879 for “Incident reporting system and method” byinventor Sloo filed Feb. 26, 1999, describes a computer-based method ofcollecting and processing incident reports received from witnesses whoobserve incidents such as criminal acts and legal violations. The methodautomates the collection and processing of the incident reports andautomatically sends the incident reports to the appropriate authority sothat the observed incidents can be acted on in an appropriate manner.For example, a witness may be equipped with a video input system such asa personal surveillance camera and a display. When the witnessencounters an incident such as a suspect committing a crime, the videoinput system would automatically recognize the suspect from the videoinput and could then display records for the suspect on the witness'shand held readout without revealing the suspect's identity. The witnesswould not need to know the identity of the suspect to observe theincident relating to the suspect. Such a system may overcome some of theproblems associated with publicly revealing personal data.

U.S. Publication 2009/0087161 for “Synthesizing a presentation of amultimedia event” by inventors Roberts, et al. filed Sep. 26, 2008,discloses a media synchronization system includes a media ingestionmodule to access a plurality of media clips received from a plurality ofclient devices, a media analysis module to determine a temporal relationbetween a first media clip from the plurality of media clips and asecond media clip from the plurality of media clips, and a contentcreation module to align the first media clip and the second media clipbased on the temporal relation, and to combine the first media clip andthe second media clip to generate the presentation. Each user whosubmits content may be assigned an identity (ID). Users may upload theirmovie clips to an ID assignment server, attaching metadata to the clipsas they upload them, or later as desired. This metadata may, forexample, include the following: Event Name, Subject, Location, Date,Timestamp, Camera ID, and Settings. In some example embodiments,additional processing may be applied as well (e.g., by the recognitionserver and/or the content analysis sub-module). Examples of suchadditional processing may include, but are not limited to, thefollowing: Face, instrument, or other image or sound recognition; Imageanalysis for bulk features like brightness, contrast, color histogram,motion level, edge level, sharpness, etc.; Measurement of (and possiblecompensation for) camera motion and shake.

U.S. Publication 2012/0282884 for “System and method for the emergencyvoice and image e-mail transmitter device” by inventor Sun filed May 5,2011, describes a voice and image e-mail transmitter device with anexternal camera attachment that is designed for emergency andsurveillance purposes is disclosed. The device converts voice signalsand photo images into digital format, which are transmitted to thenearest voice-image message receiving station from where the digitalsignal strings are parsed and converted into voice, image, or videomessage files which are attached to an e-mail and delivered to userpre-defined destination e-mail addresses and a 911 rescue team. Thee-mail also includes the caller's voice and personal information, photoimages of a security threat, device serial number, and a GPS locationmap of the caller's location. When the PSU device is initially used, theuser needs to pre-register personal information and whenever a digitalsignal string is transmitted out from the PSU device it will includethese personal information data plus a time code of the message beingsent, the PSU device's unique serial number, and the GPS generatedlocation code, etc. which will all be imbedded in the PSU e-mail.

U.S. Publication 2012/0262576 for “Method and system for a network ofmultiple live video sources” by inventors Sechrist, et al. filed Mar.15, 2012, discloses a system and a method that operate a network ofmultiple live video sources. In one embodiment, the system includes (i)a device server for communicating with one or more of the video sourceseach providing a video stream; (ii) an application server to allowcontrolled access of the network by qualified web clients; and (iii) astreaming server which, under direction of the application server,routes the video streams from the one or more video sources to thequalified web clients.

Geo-location information and contemporaneous timestamps may be embeddedin the video stream together with a signature of the encoder, providinga mechanism for self-authentication of the video stream. A signaturethat is difficult to falsify (e.g., digitally signed using anidentification code embedded in the hardware of the encoder) providesassurance of the trustworthiness of the geo-location information andtimestamps, thereby establishing reliable time and space records for therecorded events. In general, data included in the database may beroughly classified into three categories: (i) automatically collecteddata; (ii) curated data; and (iii) derivative data. Automaticallycollected data includes, for example, such data as reading fromenvironmental sensors and system operating parameters, which arecollected as a matter of course automatically. Curated data are datathat are collected from examination of the automatically collected dataor from other sources and include, for example, content-basedcategorization of the video streams. For example, detection of asignificant amount of motion at speeds typical of automobiles maysuggest that the content is “traffic.” Derivative data includes any dataresulting from analysis of the automatically collected data, the curateddata, or any combination of such data. For example, the database maymaintain a ranking of video source based on viewership or a surge inviewership over recent time period. Derivative data may be generatedautomatically or upon demand.

None of the prior art provides solutions for cloud-based analytics forsmart surveillance as provided by the present invention.

SUMMARY OF THE INVENTION

The present invention relates to cloud-based video surveillance systemproviding a way for users to set up a smart cloud-based surveillancesystem with various smart devices.

The present invention provides a cloud-based video surveillance systemincluding: at least one local computing device constructed andconfigured in network communication with at least one input capturedevice over a local network. The at least one device has at least onevisual sensor embedded. The at least one local computing device isoperable to query a network device (e.g. a router) and identify the atleast one input capture device on the same network, and connect the atleast one input capture device to a cloud platform. The at least oneinput capture device communicates input data to the cloud platform. Theplatform may provide data storage and analytics. The cloud-basedsurveillance system can be viewed on the at least one local computingdevice. The video surveillance system can also be viewed on otherauthorized computing devices locally and/or remotely, such as laptops,personal computers, tablets, smartphones and other similar devices.Especially, when the at least one computing device is powered off,authorized users can still access to the cloud-based surveillance systemvia these other authorized computing devices remotely.

These and other aspects of the present invention will become apparent tothose skilled in the art after a reading of the following description ofthe preferred embodiment when considered with the drawings, as theysupport the claimed invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an exemplary system consistent with theinvention.

FIG. 2 is a flowchart of a method for providing a cloud-basedsurveillance system of the present invention.

FIG. 3 is a schematic diagram of one embodiment of the invention.

FIG. 4 is a schematic diagram of one embodiment of the invention.

FIG. 5 is a schematic diagram of one embodiment of the invention.

FIG. 6 is a schematic diagram of a cloud-based system of the presentinvention.

FIG. 7 is another schematic diagram of a cloud-based system of thepresent invention.

DETAILED DESCRIPTION

Referring now to the drawings in general, the illustrations are for thepurpose of describing a preferred embodiment of the invention and arenot intended to limit the invention thereto.

The present invention relates to cloud-based surveillance systems andmethods for providing at least one server computer in communication witha network for providing centralized and/or distributed cloud-basedanalytics of inputs captured from remote input capture devices forproviding analyzed inputs that are stored in the cloud-based systemdatabase and accessible remotely and securely for providing security forat least one surveillance environment, surveillance event, and/orsurveillance target. Related secure wired and/or wireless networks andsystems, and methods for using them are disclosed in U.S. Publications2014/0071289, and U.S. Pat. Nos. 7,784,080, 7,719,567, 7,954,129,7,728,871, 7,730,534, 8,395,664, and 8,752,106, each of which isincorporated herein by reference in its entirety.

Although “cloud computing” can generically be applied to any software asa service or to services interfacing through the Internet, in thepresent invention, “cloud-based” computing refers to distributedcomputing among at least one server or more than one server.

The present invention provides a cloud-based video surveillance systemincluding: at least one local computing device constructed andconfigured in network communication with at least one input capturedevice over a local network. The at least one device has at least onevisual sensor embedded. The at least one local computing device isoperable to query a network device (e.g. a router) and identify the atleast one input capture device on the same network, and connect the atleast one input capture device to a cloud platform. The at least oneinput capture device communicates input data to the cloud platform. Theplatform may provide data storage and analytics. The cloud-basedsurveillance system can be viewed on the at least one local computingdevice. The video surveillance system can also be viewed on otherauthorized computing devices locally and/or remotely, such as laptops,personal computers, tablets, smartphones and other similar devices.Especially, when the at least one computing device is powered off,authorized users can still access to the cloud-based surveillance systemvia these other authorized computing devices remotely.

In one embodiment, the at least one local computing device can be a gameconsole, for example, Xbox One. The game console includes a processorand a memory. One or more video cameras can be placed anywhere within aresidential place. The game console and the one or more video camerasare connected to a local communication network via a router wired orwireless. An application program for setting up a surveillance system isinstalled on this game console. The application-program-equipped gameconsole communicates to the router and identify all the video camerasconnected to the local communication network via the router, andprovides identifiers of the one or more video cameras to a databaseaccessible by the game console and connects them to a cloud platform aswell. The one or more video cameras will then communicate videosurveillance data directly to the cloud platform. Thus, a cloud-basedvideo surveillance system is set up. A user can view the videosurveillance data in live and recorded formats from the displayconnected with the game console or remotely from other compatibledevices, such as smartphones, notebooks, tablets, and other similardevices.

A user having an Xbox login account is able to download, install andactivate the application program to the Xbox One game console system.The user can place multiple cameras in a certain local area network,each camera has its own identifier, but the user can access thesemultiple cameras with one login over the cloud platform. The system andmethod of the present invention provides for at least one game systemand corresponding account(s) are operable as video surveillance withinthe communication network after at least one camera, at least one videoinput device, or at least one input capture device (ICD) is installedand activated over the same local network. The cloud platform is thenaccessible for view and control access by one or other communicationdevice locally and/or remotely with the Xbox login account, whether theat least one game system is being played or not, i.e., if the at leastone game system is actively in use or not. Especially when the Xbox gameconsole is powered off, a user can still access to the cloud-based videosurveillance system via other computing devices with the Xbox login.

By way of example and not limitation, with the video surveillancesystem, gamers are able to play and at the same time keep an eye onother things around home, for example, keep watch on a sleeping child ina different room or a cooking pot in the kitchen, or check out who isringing doorbell. Even when the game console is powered off, the videosurveillance system is still at work. Authorized users with the Xboxlogin account can access the video surveillance system remotely viaother computing devices and receive messages and alerts. Users can alsochoose to record and store videos on the cloud platform for a certainperiod, for example, a week or a month. So the cloud platform isoperable to record and store video surveillance data, and toautomatically deliver messages and/or alerts to mobile communicationdevices and/or computers associated with the Xbox account login andauthorized user(s).

In another example, the at least one local computing device is a smartTV. A household video surveillance system can be set up by the smart TVwith an application program when one or more cameras placed within thehousehold are connected to the same local communication network as thesmart TV. Thus, a user is able to keep connected with things happeningin the surveillance area while watching TV programs. Even when the smartTV is turned off, the video surveillance system is still accessible viaother computing devices and the user can receive emails and/or alertsregarding the household video surveillance system.

Yet in another embodiment, the at least one local computing device is asmart car with a computing component and a network device. One or morecameras installed in the car are communicatively connected with thecommunication component. An application program for setting up a carvideo surveillance system may be downloaded, installed and activated onthe computing component. The computing component communicates with thenetwork device and to identify all the one or more video cameras withinthe car. Then the computing component adds identifiers of the one ormore video cameras to a cloud platform and configures the one or morevideo cameras for communicating with the cloud platform as well. Thecloud platform is operable to provide storage analytics services tosupport visual intelligence by the system.

With such a video surveillance system, a driver is able to record videosfrom different angles of the car for insurance claim issues in case anaccident happens to the car. Moreover, even when the smart car is turnedoff and no driver is in the smart car, authorized drivers can stillaccess to the car video surveillance system via smartphones, notebooks,tablets and other similar devices remotely. For example, a driver maypark the car in a parking lot and go to a shopping center, the driver isable to view video surveillance data remotely and receive alerts and/ormessages as to what happens to the car, for example an impact fromanother car or any suspicious activities around the car.

With the development of IoTV, various devices embedded with visualsensors can be used to monitor and control the mechanical, electricaland electronic systems used in various types of buildings (e.g., publicand private, industrial, institutions, or residential). Home automationsystems, like other building automation systems, are typically used tocontrol lighting, heating, ventilation, air conditioning, appliances,communication systems, entertainment and home security devices toimprove convenience, comfort, energy efficiency, and security.

Within a residential location, various smart appliances havecommunication capability. For example, a smart refrigerator has embeddedvisual sensors are able to residents if what groceries they still have.Some smart refrigerators can even tell what items they have and whatitems they need to buy and send reminders to the residents via phonemessages and/or emails. For example, a doorbell embedded with a visualsensor can record who come to the front door or the view field of thevisual sensor. Such smart visual-sensor-equipped doorbells even haveadvanced analytics functions such as facial recognition, eventrecognition, etc., and send alerts to residents who are not home viaphone message and/or email. If a resident has multiple such smartdevices, it will be more convenient to connect and access all thesesmart devices under one account.

In one embodiment, a local computing device with a specific applicationprogram installed query a network device (e.g. a router) over a homenetwork and finds all the smart devices connected to home network viathe network device. The local computing device connects all the smartdevice to a cloud platform. Thus, a smart home surveillance system isset up. A resident can access the all these smart devices under oneauthorized account via the local computing device or other remotecomputing device. The local computing device and other remote computingdevices can be personal computers, laptops, smartphones, tablets, andother similar devices. Specifically, when the local computing device ispowered off, the smart home surveillance system is still accessiblethrough other authorized computing devices.

In one embodiment, one authorized account is associated with onesurveillance system and the cloud platform. Each of the visual sensorsembedded on each of the devices has a unique identifier, each of theconnected device also has a unique identifier. All of the individualdevice accounts and visual sensor accounts are added under theauthorized account. Thus, all the devices and visual sensors areaccessed with one login.

Video cameras used in the current invention are either generic videocamera devices, or smart Input Capture Devices (ICDs) which haveembedded cameras, including but not limited to smart phones, tabletcomputers, mobile communication devices, portable computers, wearablecomputers and wearable input capture and communication devices. Videocameras used in the current invention can be wired or wireless. In anycase, the video sensors in a smart ICD have unique identifiers and allthe smart ICDs have unique identifiers as well.

In one embodiment, the video cameras further include additionalfunctionality, for example two-way audio, night vision, temperaturesensitive video capture, etc. Also preferably, the video cameras in thisinvention are equipped with a time-stamp function that embeds orassociates a date and time data with an image or video. Authentication,messages, and/or alerts from the cloud platform also provide anassociated date and time data in one embodiment of the presentinvention.

In some embodiments, the video cameras may provide geographic locationinformation and/or Global Positioning System (GPS) information to beembedded within videos and images sent from the video cameras over thenetwork to the cloud platform. Similarly, the messages and/or alertsfrom the cloud platform also preferably have GPS information.

Video and/or image inputs are indexed, organized, stored, and availablefor access by authorized users via the network through a website orportal or Application Program Interface (API). The at least one localcomputing device and other authorized computing devices are preferablyregistered with the system through an API, an app, or softwareapplication for accessing the video surveillance system.

The at least one local computing device and other authorized computingdevices are selected from game consoles (e.g. Xbox), smart TVs, smartcars, smart appliances, smart environmental detection devices, smartenvironmental monitoring and control devices, personal computers,laptops, tablets, smartphones, and/or other similar smart devices.

The cloud platform provides access to video surveillance data in bothlive and recorded formats. A user may select an amount of video and anamount of time period for storage on the cloud platform. In commercialembodiments, payment alternatives providing for varied levels of accessto the system, data storage, analytics, intelligence, reports, etc. areincluded with user profiles that are stored by the system.

The cloud platform also provides visual intelligence for the videosurveillance system. Basic analytics may be provided for free, forexample motion detection. Authorized users will receive an alert ormessage when there is a condition detected. The user may also selectcertain types of advanced cloud-based analytics service at certainprices, for example, facial recognition, event recognition, predictivewarning, voice reminders, etc. The cloud platform will then sendmessages and/or alerts when there are changes in state.

Optionally, the software application and/or the computing devicesautomatically verifies and authenticates a user, for example usingbiometric authentication, such as facial recognition, fingerprint, etc.,and/or using a user identification and passcode or personalidentification number, or other authentication mechanisms.

In preferred embodiments of the present invention, an applicationprogram is downloaded, installed and activated on the at least one localcomputing device for setting up a video surveillance system. A graphicaluser interface is provided by the application program on the at leastone local computing device and other authorized computing devices withtheir corresponding displays to provide secure, authorized access to thevideo and/or image contents associated with the video surveillancesystem.

Also, preferably, the application program on the at least one localcomputing device and other remote computing devices provide an automatedsharing feature that provides for single click select and activation ofmedia sharing of the selected inputs captured. In one embodiment, thesingle click select and activation of media sharing of the selectedvideo and/or image provides time and identity information.

In some embodiments, the smart devices embedded with visual sensorsprovide some intelligent functions locally besides recording andstreaming videos. For example, a doorbell with embedded visual sensorhas the ability to recognize faces and events in the view field of thevisual sensor. For example, if the visual sensor embedded in an ovenfinds out the oven light is left on after the cooking is done, it canturn off the oven light itself. The cloud platform is able to doperformance prediction and generate commands and provide centralizedcoordination and other intelligence functions among all the smartdevices within such a surveillance system.

Video cameras and smart devices are equipped with wired and/or wirelesscommunication capabilities to a local router. ZigBee, Wi-Fi, or meshnetwork technologies may be applied to certain surveillance system. Inone embodiment, all the devices are communicating with the cloudplatform, there is no communication between different devices with asurveillance system. In another embodiment, there is machine-to-machinecommunication between devices on top of a centralized communication tothe cloud platform. In this situation, the smart devices or ICDs havelocal intelligent analytics and communicate command and control to otherinput capture devices within a surveillance system over communicationnetwork. For example, if a thermostat with an embedded visual sensorfinds out the light is on during the day when no one is at home, it canturn off the light. For example, if the doorbell embedded with visualsensors finds out authorized residents are walking towards the door withmany grocery bags, it then unlocks the door for the authorizedresidents.

The network device can be a wireless/wired router or an antenna. The atleast one local computing device equipped with the application programhas polling capabilities and is enabled to access the network device.The network device has records of all connected device, the at leastcomputing device is operable to poll and identify all the devices thatare connected the network device over a communication network. Thepolling, querying and identifying capabilities are disclosed in U.S.Pat. No. 7,508,418, which in incorporated herein by reference in itsentirety. The communication network can be a Wide Area Network (WAN), aLocal Area Network (LAN), or a Personal Area Network (PAN).

The present invention provides a cloud-computing surveillance systemincluding: at least one server computer having a processor and a memory,constructed and configured in network-based communication with amultiplicity of remote input devices having input capture mechanisms;inputs captured by the remote input devices transmitted within a securemessaging communicated over the network; wherein the inputs arereceived, authenticated, and indexed by the at least one server computerand stored in a corresponding database; wherein the inputs are processedand analyzed based upon at least one profile for a surveillanceenvironment, a surveillance event, and/or a surveillance target, forproviding a near-real-time analysis of the inputs to determine a statusof security. The at least one profile associated with the surveillanceenvironment, surveillance event, and/or surveillance target may includesecurity level (low, medium, high), alert level, time interval forreview for change, authorized remote input device and/or userinformation, and combinations thereof. The status is selected from:normal, questionable, alert, urgent, disaster, injury, and anydescriptor or indicator of the level and condition of the environment,event, and/or target compared with predetermined conditions.

The system may further include a priority and a profile associated withthe inputs for automatically associating the inputs with thecorresponding surveillance environment, surveillance event, and/orsurveillance target. The profile associated with the inputs may includeuser and/or owner identifier, equipment identifier, communicationsecurity level, and combinations thereof. In one embodiment, the securemessaging includes internet protocol (IP) messaging of data packet(s)including the inputs, and may further include encryption, digitalfingerprinting, watermarking, media hashes, and combinations thereof. Asdescribed in the following detailed description of the invention, theinputs are selected from images, audio, and/or video; more particularly,the input is selected from live streaming video, real-time images and/oraudio, previously recorded video, previously captured images and/oraudio, and combinations thereof. The remote input devices include mobilephones, smart phones, tablet computers, portable computers, mobilecommunication devices, wearable input capture devices, and/or securitycameras. By way of example and not limitation, a wearable input capturedevice may be removable, portable devices such as eyewear (like GoogleGlass), headwear, wristwear, etc.

The analysis is performed by a virtualized or cloud-based computingsystem and provides for remote access of analyzed inputs, and involvesat least one rules engine for transforming individual inputs intoanalyzed content. The analyzed content may include inputs from more thanone remote input device. Additionally, the analyzed content may begenerated by transforming the original inputs by the at least one servercomputer automatically assembling input fragments into an integratedcontent file, and wherein the original input is stored and associatedwith the integrated content file.

In one embodiment of the present invention, the authentication includesconfirmation of global positioning system (GPS) location of each of theremote input devices providing inputs and matching the GPS location withcorresponding at least one predetermined surveillance environment,surveillance event, and/or surveillance target. Preferably, the analysisincludes authentication of the input device with a deviceidentification, a user identification, a geographic location, and a timeassociated with the input and the predetermined surveillanceenvironment, surveillance event, and/or surveillance target.

At the at least one server computer, the authenticated inputs areautomatically tagged, combined, grouped, edited, and analyzed by thecloud-based system according to the predetermined surveillanceenvironment, surveillance event, and/or surveillance target. Also, theinput is verified by authenticating the at least one input device and/orits corresponding user and the input is analyzed to confirm that therehas been no alteration, editing, and/or modification to the input priorto its receipt by the at least one server computer.

The present invention also provides methods for the system described inthe foregoing, including the steps of: providing a cloud-based orvirtualized computing system having at least one server computer with aprocessor and a memory, constructed and configured in network-basedcommunication with a multiplicity of remote input devices having inputcapture mechanisms; receiving by the at least one server computer inputsfrom the remote input devices transmitted within a secure messagingcommunicated over the network; authenticating the inputs; indexing theinputs by the at least one server computer; and storing the inputs in acorresponding database; processing and analyzing the inputs by the atleast one server computer using at least one profile for a surveillanceenvironment, a surveillance event, and/or a surveillance target, forproviding a near-real-time analysis of the inputs to determine a statusof security. Additional steps may include: providing a priority for thesecure messaging; analyzing inputs from more than one remote inputdevice in near real time to provide social security surveillance of thesurveillance environment, surveillance event, and/or surveillancetarget; and/or automatically assembling input fragments into anintegrated content file, and wherein the original input is stored andassociated with the integrated content file. Also, preferably, theauthenticating step includes automatic authentication of the inputdevice and/or its user based upon the combination of a deviceidentification, a user identification, a geographic location, and a timeassociated with the input and the predetermined surveillanceenvironment, surveillance event, and/or surveillance target.

The present invention systems and methods include a surveillance systemfor providing automated cloud-based analytics that allows for uploadingof captured inputs, authentication of the inputs, and analysis of theinputs to provide real- or near real-time surveillance of a surveillanceenvironment, surveillance event, and/or surveillance target. Thesurveillance and visual intelligence systems and methods of the presentinvention include a combination of several key features including inputauthentication, time, and automated cloud-based analytics relating tothe inputs and the surveillance environment, surveillance event, and/orsurveillance target.

The authentication is provided with device and/or user with locationwherein the input devices provide information including geographiclocation information and/or global positioning system (GPS) informationto be embedded within images and videos and/or included in the messagingfrom the input devices over the network to the at least one servercomputer. Additionally, overlay and other techniques may also be usedduring upload of content, such as, by way of example and not limitation,TDOA, AIA, and RF fingerprinting technologies.

Preferably, the input devices are equipped with a time-stamp functionthat embeds a date and time into an image or video for laterauthentication, or their messaging provides a date and time associatedwith the inputs, including images, and/or video.

Additionally, the authentication of users and/or devices through theevaluation of uploaded content, including stenographic techniques suchas digital fingerprinting and watermarking, or user-verificationtechniques such as login or CAPTCHA technologies and biometric scanning.

While some content is considered verified by authenticating a user ordevice, additional analytics may be performed by the cloud-based systemto establish that content has not been modified from its originalsources, such as through the use of media hashes. Additionally, afterreceiving and authenticating multiple sources of information, analyticsmay allow for the inputs to be aggregated, tagged, combined, edited,and/or grouped.

Although in the prior art, content-based analytics is used in CCTVsettings and when verifying that digital content has been unaltered orauthenticating a content's source (e.g., copyrighted music, images andvideos), it has not been used for distributed, cloud-based socialsurveillance allowing for a multiplicity of inputs from remote inputdevices to at least one server computer for analysis of the inputs basedupon a predetermined surveillance environment, surveillance event,and/or surveillance target, and more particularly for securitysurveillance.

Notably, specialized pre-registered devices are not required, butinstead the present invention incorporates distributed, and potentiallyunknown devices, so long as the user, time and location correspond tothe predetermined surveillance environment, surveillance event, and/orsurveillance target.

Systems and methods of the present invention provide for a multiplicityof remote input devices, by way of example and not limitation, includingcommercially available devices such as Google glass or glasses orheadwear having input capture mechanisms and mobile communicationcapability, mobile smart phones, cellular phones, tablet computers,gaming devices such as an Xbox Kinect controller, so long as the inputdevice is constructed and configured to capture and share or transmitvideo and/or images associated with location data, direction, etc. andowners/users with the cloud-based surveillance system. The inputinformation is stored on at least one server computer, in a centralizedand/or virtualized central manner, and the input information is indexed,organized, stored, and available for access by authorized users via thenetwork through a website or portal or API. The input device ispreferably registered with the system through an app or softwareapplication associated with the remote or distributed input devices.While preregistration is not required for the inputs to be associatedwith at least one surveillance environment, surveillance event, and/orsurveillance target, all inputs are required to be authenticated by thesystem based upon the input device, the input device user, and/orcorresponding identification and/or association with the surveillanceenvironment, surveillance event, and/or surveillance target. By way ofexample and not limitation, a video input is transmitted by a remoteinput device with an email including the video input as a mediaattachment within the message; the cloud-based system and its at leastone server computer receives the email message, authenticates the emailaddress associated with the device and/or user, and accepts the video.Also the same is provided with MMS or text messaging with video and/oraudio and/or image.

In one embodiment of the present invention, method steps include:providing the system as described hereinabove; providing a softwareapplication operating on a remote input device for capturing at leastone input including an image, a video, and/or an audio input; activatingthe software application; capturing the at least one input including animage, a video, and/or an audio input; automatically and/or manuallyincluding structural and/or descriptive metadata, including but notlimited to unique identifying indicia associated with the input, time,location or geographic information, text and/or audio notationassociated with the input, priority flag or indicator, and combinationsthereof.

Optionally, the software application and/or the remote input deviceautomatically verifies and authenticates the user of the remote inputdevice, for example using biometric authentication such as facialrecognition, fingerprint, etc., and/or using a user identification andpasscode or personal identification number, or other authenticationmechanisms. Preferably, the authentication information is included withthe metadata corresponding to the input(s) and associated therewith as acomposite input, and the software application and/or the remote inputdevice automatically transmits the composite input over the network tothe cloud-based system and the at least one server computer thereon andis saved in at least one database. In preferred embodiments of thepresent invention, a user interface is provided on the remote inputdevice(s) or distributed computer device(s) and their correspondingdisplays to provide secure, authorized access to the composite inputand/or to all inputs associated with predetermined surveillanceenvironment, surveillance event, and/or surveillance target stored inthe cloud database.

Also, preferably, the software application on the remote input deviceprovides an automated sharing feature that provides for single clickselect and activation of media sharing of the selected inputs captured.In one embodiment, the single click select and activation of mediasharing of the selected inputs captured on that remote input deviceprovides for automatic association of the shared media with at least oneemail address corresponding to the user and the remote input device.

FIG. 1 illustrates a block diagram of an exemplary system 100 consistentwith the invention. As shown in FIG. 1, exemplary system 100 maycomprises two ICDs 101, 102, a cloud platform 103 and a computing device104. The cloud platform 103 are constructed and configured in networkcommunication with the two ICDs 101, 102 and the user device 104. Thetwo ICDs each have a visual sensor 105, 106, respectively. The cloudplatform 103 has a processor 107 and a memory 108. The computing devicehas a display with a user interface 109.

FIG. 2 is a flowchart 200 illustrating a method for providing acloud-based surveillance system in the present invention. The methodcomprises (201) communicatively connecting a local computing device andan input capture device to a local communication network via a networkdevice. The method further comprises (202) the local computing devicequerying the network device and identifying the input capture device.The method further comprises (203) the local computing deviceconfiguring the input capture device for communicating with a cloudplatform and (204) the input capture device communicating visual data tothe cloud platform. The method further comprises (205) the localcomputing device and other authorized computing devices accessing to thevisual data on the cloud platform and (206) other authorized computingdevices accessing to the visual data on the cloud platform when thelocal computing device is powered off.

FIGS. 3-5 illustrate schematic diagrams of different embodiments of thepresent invention; like reference indicators are used throughout themultiple figures for the same or similar elements, as appropriate. FIG.3 shows one embodiment of a cloud-based video surveillance system 300.The embodiment shows a CPU processor and/or server computer 120 innetwork-based communication with at least one database 130 and at leastone geographically redundant database 140. The server computer 120 isconnected to a network 110, a communications (wired and/or wireless)router 180, communications tower 160, and a user device 150 are alsoconnected to the network 110. A user device 170 is connected to thenetwork 110 via the communication tower 160. A user device 190 and twoICDs 310 and 320 are connected to the router 180 in a local area networkvia Wi-Fi wireless 601, cellular wireless 602, or Bluetooth wireless603. Each of the two ICDs may include image capture 610, video capture620, audio capture 630, text and audio note 640, and/or geo-location 650technologies, each technology capable of collecting data for upload tothe network 110 and storage on the databases 130, 140. As the userdevice 190 may also contain identity technologies 920, such as facial,fingerprint and/or retina recognition, both databases 130, 140 mayinclude identity database for validating fingerprints, facialrecognition, and/or retina recognition. User devices 150 and 170, beingany computer, tablet, smartphone, or similar device, permits user accessto the data, video, image, and audio storage on the cloud.

FIG. 4 illustrates another embodiment 400 of a cloud-based videosurveillance system providing for the components shown. A communicationsrouter 180 is connected with the network via communication tower 160.

FIG. 5 illustrates another cloud-based video surveillance system 500with the components shown, including a software application or app on acomputing device having a graphic user interface (GUI) providing for alive viewing area on the device and function buttons, virtual buttons(i.e., touch-activated, near-touch-activated, etc.) of record, notes,and send, associated with input capture devices 190.

Referring now to FIG. 6, a schematic diagram 600 illustrating avirtualized computing network used in of one embodiment of the inventionfor automated systems and methods is shown. As illustrated, componentsof the systems and methods include the following components andsub-components, all constructed and configured for network-basedcommunication, and further including data processing and storage. Asillustrated in FIG. 6, a basic schematic of some of the key componentsof a financial settlement system according to the present invention areshown. The system 600 comprises a server 210 with a processing unit 211.The server 210 is constructed, configured and coupled to enablecommunication over a network 250. The server provides for userinterconnection with the server over the network using a personalcomputer (PC) 240 positioned remotely from the server, the personalcomputer has instructions 247 stored in memory 246. There are othernecessary components in the PC 240, for example, a CPU 244, BUS 242,Input/Output (“I/O”) port 248, and an Output (“O”) port 249.Furthermore, the system is operable for a multiplicity of remotepersonal computers or terminals 260, 270, having operating systems 269,279. For example, a client/server architecture is shown. Alternatively,a user may interconnect through the network 250 using a user device suchas a personal digital assistant (PDA), mobile communication device, suchas by way of example and not limitation, a mobile phone, a cell phone,smart phone, laptop computer, netbook, a terminal, or any othercomputing device suitable for network connection. Also, alternativearchitectures may be used instead of the client/server architecture. Forexample, a PC network, or other suitable architecture may be used. Thenetwork 250 may be the Internet, an intranet, or any other networksuitable for searching, obtaining, and/or using information and/orcommunications. The system of the present invention further includes anoperating system 212 installed and running on the server 210, enablingserver 210 to communicate through network 250 with the remotedistributed user devices. The operating system may be any operatingsystem known in the art that is suitable for network communication asdescribed herein below. Data storage 220 may house an operating system222, memory 224, and programs 226.

Additionally or alternatively to FIG. 6, FIG. 7 is a schematic diagramof an embodiment of the invention illustrating a computer system,generally described as 700, having a network 810 and a plurality ofcomputing devices 820, 830, 840. In one embodiment of the invention, thecomputer system 800 includes a cloud-based network 810 for distributedcommunication via the network's wireless communication antenna 812 andprocessing by a plurality of mobile communication computing devices 830.In another embodiment of the invention, the computer system 800 is avirtualized computing system capable of executing any or all aspects ofsoftware and/or application components presented herein on the computingdevices 820, 830, 840. In certain aspects, the computer system 700 maybe implemented using hardware or a combination of software and hardware,either in a dedicated computing device, or integrated into anotherentity, or distributed across multiple entities or computing devices.

By way of example, and not limitation, the computing devices 820, 830,840 are intended to represent various forms of digital computers andmobile devices, such as a server, blade server, mainframe, mobile phone,a personal digital assistant (PDA), a smart phone, a desktop computer, anetbook computer, a tablet computer, a workstation, a laptop, and othersimilar computing devices. The components shown here, their connectionsand relationships, and their functions, are meant to be exemplary only,and are not meant to limit implementations of the invention describedand/or claimed in this document.

In one embodiment, the user device 820 includes components such as aprocessor 860, a system memory 862 having a random access memory (RAM)864 and a read-only memory (ROM) 866, and a user bus 868 that couplesthe memory 862 to the processor 860. In another embodiment, thecomputing device 830 may additionally include components such as astorage device 890 for storing the operating system 892 and one or moreapplication programs 894, a network interface unit 896, and/or aninput/output controller 898. Each of the components may be coupled toeach other through at least one bus 868. The input/output controller 898may receive and process input from, or provide output to, a number ofother devices 899, including, but not limited to, alphanumeric inputdevices, mice, electronic styluses, display units, touch screens, signalgeneration devices (e.g., speakers) or printers.

By way of example, and not limitation, the processor 860 may be ageneral-purpose microprocessor (e.g., a central processing unit (CPU)),a graphics processing unit (GPU), a microcontroller, a Digital SignalProcessor (DSP), an Application Specific Integrated Circuit (ASIC), aField Programmable Gate Array (FPGA), a Programmable Logic Device (PLD),a controller, a state machine, gated or transistor logic, discretehardware components, or any other suitable entity or combinationsthereof that can perform calculations, process instructions forexecution, and/or other manipulations of information.

In another implementation, shown in FIG. 7, a computing device 840 mayuse multiple processors 860 and/or multiple buses 868, as appropriate,along with multiple memories 862 of multiple types (e.g., a combinationof a DSP and a microprocessor, a plurality of microprocessors, one ormore microprocessors in conjunction with a DSP core).

Also, multiple computing devices may be connected, with each deviceproviding portions of the necessary operations (e.g., a server bank, agroup of blade servers, or a multi-processor system). Alternatively,some steps or methods may be performed by circuitry that is specific toa given function.

According to various embodiments, the computer system 700 may operate ina networked environment using logical connections to local and/or remotecomputing devices 820, 830, 840, 850 through a network 810. A computingdevice 830 may connect to a network 810 through a network interface unit896 connected to the bus 868. Computing devices may communicatecommunication media through wired networks, direct-wired connections orwirelessly such as acoustic, RF or infrared through a wirelesscommunication antenna 897 in communication with the network's wirelesscommunication antenna 812 and the network interface unit 896, which mayinclude digital signal processing circuitry when necessary. The networkinterface unit 896 may provide for communications under various modes orprotocols.

In one or more exemplary aspects, the instructions may be implemented inhardware, software, firmware, or any combinations thereof. A computerreadable medium may provide volatile or non-volatile storage for one ormore sets of instructions, such as operating systems, data structures,program modules, applications or other data embodying any one or more ofthe methodologies or functions described herein. The computer readablemedium may include the memory 862, the processor 860, and/or the storagemedia 890 and may be a single medium or multiple media (e.g., acentralized or distributed computer system) that store the one or moresets of instructions 900. Non-transitory computer readable mediaincludes all computer readable media, with the sole exception being atransitory, propagating signal per se. The instructions 900 may furtherbe transmitted or received over the network 810 via the networkinterface unit 896 as communication media, which may include a modulateddata signal such as a carrier wave or other transport mechanism andincludes any delivery media. The term “modulated data signal” means asignal that has one or more of its characteristics changed or set in amanner as to encode information in the signal.

Storage devices 890 and memory 862 include, but are not limited to,volatile and non-volatile media such as cache, RAM, ROM, EPROM, EEPROM,FLASH memory or other solid state memory technology, disks or discs(e.g., digital versatile disks (DVD), HD-DVD, BLU-RAY, compact disc(CD), CD-ROM, floppy disc) or other optical storage, magnetic cassettes,magnetic tape, magnetic disk storage or other magnetic storage devices,or any other medium that can be used to store the computer readableinstructions and which can be accessed by the computer system 700.

It is also contemplated that the computer system 700 may not include allof the components shown in FIG. 7, may include other components that arenot explicitly shown in FIG. 7, or may utilize an architecturecompletely different than that shown in FIG. 7. The various illustrativelogical blocks, modules, elements, circuits, and algorithms described inconnection with the embodiments disclosed herein may be implemented aselectronic hardware, computer software, or combinations of both. Toclearly illustrate this interchangeability of hardware and software,various illustrative components, blocks, modules, circuits, and stepshave been described above generally in terms of their functionality.Whether such functionality is implemented as hardware or softwaredepends upon the particular application and design constraints imposedon the overall system. Skilled artisans may implement the describedfunctionality in varying ways for each particular application (e.g.,arranged in a different order or partitioned in a different way), butsuch implementation decisions should not be interpreted as causing adeparture from the scope of the present invention.

Certain modifications and improvements will occur to those skilled inthe art upon a reading of the foregoing description. By way of exampleand not limitation, the present invention systems and methods mayfurther include automated web-based searching to identify and analyzesimilar images and/or videos (or content, individuals, objects, andcombinations thereof in the images and/or videos) from social websitesor social media postings to associate, link, supplement and/or matchwith the at least one input authenticated and received by thecloud-based server(s) and corresponding to a surveillance environment, asurveillance event, and/or a surveillance target within a predeterminedtimeframe. The above-mentioned examples are provided to serve thepurpose of clarifying the aspects of the invention and it will beapparent to one skilled in the art that they do not serve to limit thescope of the invention. All modifications and improvements have beendeleted herein for the sake of conciseness and readability but areproperly within the scope of the present invention.

The invention claimed is:
 1. A method for connecting an input capturedevice to a cloud platform, comprising: communicatively connecting alocal computing device and the input capture device to a localcommunication network via a network device, wherein the input capturedevice includes one or more environment sensors; the local computingdevice querying the network device; the local computing deviceidentifying the input capture device; the local computing deviceconfiguring the input capture device for communicating with the cloudplatform, wherein the input capture device is configured to send, via acommunication path, environmental data to the cloud platform, whereinthe local computing device is not part of the communication path; theinput capture device communicating the environment data to the cloudplatform, wherein the environmental data includes one or more of audiodata, video data, motion data, or still images; wherein the environmentdata on the cloud platform are accessible by the local computing devicelocally and/or other authorized computing devices; and wherein theenvironment data on the cloud platform are accessible by the otherauthorized computing devices even when the local computing device ispowered off.
 2. The method of claim 1, wherein the local computingdevice and the other authorized computing devices are selected from thegroup consisting of: personal computers, laptops, tablets, smartphones,game consoles, and smart TVs.
 3. The method of claim 1, wherein theinput capture device is selected from the group consisting of: doorbellswith embedded cameras, refrigerators with embedded cameras, ovens withembedded cameras, thermostats with embedded cameras, and cars withembedded cameras.
 4. The method of claim 1, further comprisinginstalling and activating an application program associated with thecloud platform on the local computing device.
 5. The method of claim 4,wherein the application program has a cloud-based account associatedwith the cloud platform.
 6. The method of claim 5, further comprisingaccessing the input capture device over the cloud platform with thecloud-based account.
 7. The method of claim 5, wherein each of theenvironment sensors of the input capture device has a unique identifier,and wherein each of at least one environment sensor is accessible underthe cloud-based account associated with the cloud platform.
 8. Themethod of claim 1, further comprising providing cloud-based videoanalytics by the cloud platform.
 9. The method of claim 1, furthercomprising storing visual data from the input capture device on thecloud platform.
 10. The method of claim 9, further comprising selectinga time period for data storage on the cloud platform.
 11. A system forcloud-based surveillance, comprising: an input capture device with atleast one environment sensor and configured to communicate with a localcomputing device; wherein the input capture device and the localcomputing device are communicatively connected to a local communicationnetwork via a network device; wherein the local computing device isoperable to: query the network device; identify the input capturedevice; and configure the input capture device for communicating with acloud platform, wherein the input capture device is operable tocommunicate, via a communication path, environment data obtained by theinput capture device to the cloud platform, wherein the local computingdevice is not part of the communication path, wherein the environmentdata includes one or more of visual data or audio data originated fromthe input capture device, wherein the environment data on the cloudplatform are accessible by the local computing device locally and/or byother authorized computing devices, and wherein the environment data onthe cloud platform are accessible by other authorized computing deviceseven when the local computing device is powered off.
 12. The system ofclaim 11, wherein the local computing device and the other authorizedcomputing devices are selected from the group consisting of: personalcomputers, laptops, tablets, smartphones, game consoles, and smart TVs.13. The system of claim 11, wherein the input capture device is selectedfrom the group consisting of: doorbells with embedded cameras,refrigerators with embedded cameras, ovens with embedded cameras,thermostats with embedded cameras, and cars with embedded cameras. 14.The system of claim 11, wherein an application program associated withthe cloud platform is installed and activated on the local computingdevice.
 15. The system of claim 14, wherein the application program hasa cloud-based account associated with the cloud platform.
 16. The systemof claim 15, wherein the local computing device and other authorizeddevices access to the device over the cloud platform with thecloud-based account.
 17. The system of claim 15, wherein each of the atleast one environment sensor on the input capture device has a uniqueidentifier, and wherein each of the at least one environment sensor isaccessible under the cloud-based account associated with the cloudplatform.
 18. The system of claim 11, wherein the cloud platform provideanalytics for the visual data.
 19. The system of claim 11, wherein thecloud platform provides storage for the visual data.
 20. The system ofclaim 19, wherein a time period for data storage over the cloud platformis selectable.
 21. The method of claim 1, wherein the cloud platformincludes one or more remote data storage devices.
 22. The method ofclaim 1, wherein configuring the input capture device for communicatingwith the cloud platform includes sending instructions for establishingcommunication between the input capture device and the cloud platformfrom the local computing device to the input capture device via thelocal communication network.
 23. The method of claim 1, wherein theenvironmental data includes audio data.
 24. The method of claim 1,wherein the environmental data includes video data.
 25. The method ofclaim 1, wherein the environmental data includes motion data.
 26. Themethod of claim 1, wherein the environmental data includes still images.27. The system of claim 11, wherein one or more applications unrelatedto connecting the input capture device to the cloud platform areinstalled on the local computing device.